When a digital image is taken under relatively low light conditions, for example, at dusk or nighttime, noise (also referred to as image noise) is often present in the resulting image. The image noise may present as a random variation in brightness or color information that results from, for example, a sensor (e.g., a charge-coupled device (CCD) sensor, a complementary-metal-oxide semiconductor (CMOS) sensor) and/or related circuitry of a capture device (e.g., a digital camera, a cell phone camera). Due to the lower photon count present in low light photography (e.g., due to fewer photons being received and/or measured by the sensor of the capture device), additional noise is generated by the capture device. The image noise is an undesirable component of the resultant image and leads to lower-quality images.
Generally, image noise includes two components-Gaussian noise and Poisson noise. Gaussian noise is the more typical type of noise (e.g., digital noise), often the result of circuit noise in the capture device. Poisson noise is less prevalent because it occurs more frequently (e.g., is more prevalent or noticeable) when an underlying signal (e.g., a low-light image signal) is relatively weak and the sensor response is quantized. In higher light conditions, the Poisson noise tends towards a normal distribution, such that Gaussian noise is more prevalent than Poisson noise in most images (e.g., in well-lit images).
When an image is captured in relatively low light conditions, the image noise has two components—a signal dependent component that may be modeled as Poisson noise distributed according to a rate of arrival of photons on a camera sensor (e.g., a CCD sensor or a CMOS sensor) and a signal independent component that may be modeled as Gaussian noise and is due to stationary disturbances in the image. The effective noise contains the two image noise components and can be referred to as “Poisson-Gaussian noise.” Due to the signal-dependent noise component, the noise variance of the effective noise is not constant but varies with the expectation of the image's pixel values.
One current method of removing (or reducing) Poisson-Gaussian noise in a digital image (e.g., a digital image signal) includes applying a variance-stabilizing transform (also referred to as “VST”), such as an Anscombe or Generalized Anscombe transform, to produce a digital signal having a noise component with a constant variance that is independent of the value of the input signal, which can be considered to be similar to additive Gaussian noise (e.g., additive white Gaussian noise (AWGN) with unitary variance). That is, the VST is applied to the digital image to transform the Poisson noise, whose variance is equal to its mean and hence depends on the underlying pixel value, to a noise with constant variance. Then, the transformed noise is removed (or substantially removed) by using a denoising algorithm, which can efficiently remove noise, such as AWGN, which has constant variance, such as by block-matching and 3D filtering (also referred to as “BM3D”). Lastly, an inverse variance-stabilizing transform (also referred to as “IVST”), such as an inverse Anscombe transform, is applied to the denoised digital image to transform it back to its original domain. Taken together, this method of removing or reducing Poisson-Gaussian noise from an image may be referred to as “VST-BM3D-IVST.”
However, the VST-BM3D-IVST method is sensitive to the forward transformation used in the first step, requiring that an unbiased inverse of the forward transformation is available to return the image to its original domain. Further, performance of the VST-BM3D-IVST method is relatively poor at very low intensity values, such as the case with images taken at a very low light level.
Some recently-attempted improvements to the VST-BM3D-IVST method include using iterative filtering of combinations of a noisy image with progressively refined (or filtered) images, but these improvements greatly increase the complexity of the VST-BM3D-IVST method and may not be suitable for use with relatively low-power (e.g., low processing power) mobile devices or the like.
Further, one recent deep learning approach to denoising low light images, called DeNoiseNet, attempts to learn a transformation from a Poisson noisy image (e.g., an image having a relatively high amount of Poisson noise) to a clean (denoised) image. This method, however, fails to consider noise variance stabilization.